The present invention relates to image encoding devices and image encoding methods and is applicable to an image encoding device and an image encoding method in which image encoding is performed separately on a region-of-interest and a non-region-of-interest.
In recent years, surveillance cameras have become popular, and higher frame rates, higher resolutions, and multi-aspects are also desired. However, higher frame rates, higher resolutions, and multi-aspects cause an increase in the volume of data of moving images, thus leading to an increase in communication costs and storage costs. In order to alleviate this problem, for example, there has been proposed a method of detecting a facial region from a moving image of a person and allocating a large number of bits for compressing the facial region.
JP 2010-193441A discloses a method of allocating a large number of bits to a facial region. More specifically, the technology described in JP 2010-193441A proposes a method of allocating different numbers of bits to a region-of-interest and a non-region-of-interest in a moving image so as to decide quantization parameters QP in a system that reduces the encoding amount.
Specifically, the technology described in JP 2010-193441A involves calculating, for each facial region, the area size of the facial region and reducing the difference in qualities of the facial region and a non-facial region as the facial region becomes larger in size. This solves the problem of significant deterioration in the quality of the non-facial region, which is caused as a result of using most of the encoding amount for the facial region when the facial region is large in size.
FIG. 2 is a configuration diagram illustrating the internal configuration of a conventional image encoding system 80. In FIG. 2, the image encoding system 80 receives an input image and pixels-of-interest as input data and outputs a stream.
The image encoding system 80 has a block-of-interest determining unit 81 that outputs a block-of-interest including the input pixels-of-interest, a region-of-interest area calculator 82 that receives the block-of-interest and calculates and outputs the area of a region-of-interest, a compression ratio controller 83 that determines an offset from the area of the region-of-interest, a rate controller 84 that decides a region-of-interest QP and a non-region-of-interest QP from a target bit rate and the offset and outputs the region-of-interest QP and the non-region-of-interest QP, and a compressor 85 that compresses the input image in accordance with a compression method, such as JPEG, H.264, or H.265, based on the block-of-interest, the region-of-interest QP, and the non-region-of-interest QP and outputs a stream.
The rate controller 84 has a QP controller 811 that generates a non-region-of-interest QP from the target bit rate, and a region-of-interest QP calculator 812 that generates a region-of-interest QP from the non-region-of-interest QP and the offset.
An offset is the difference between the value of the region-of-interest QP and the value of the non-region-of-interest QP.
The rate controller 84 adds and subtracts the offset to and from the quantization parameters decide by the QP controller 811 so as to generate the region-of-interest QP and the non-region-of-interest QP.
This example corresponds to a case where the rate controller 84 outputs a value obtained by subtracting the offset from the non-region-of-interest QP decide by the QP controller 811 as the region-of-interest QP. However, there is another conceivable variation in which, for example, the rate controller 84 outputs a value obtained by adding the offset to the region-of-interest QP decide by the QP controller 811 as the non-region-of-interest QP.
Although a detailed embodiment of a QP controller is not clearly specified in particular in the technology described in JP 2010-193441A, for example, a conceivable method involves performing control based on feedback (FB) information output from the compressor 85, as illustrated in FIG. 2.
In this case, FB information is, for example, the quantization parameter used and the encoding amount at that time.
Specifically, the following two control methods are conceivable.
In the first control method, the QP controller 811 compares the encoding amount for the immediately-preceding frame with the target bit rate and increases the non-region-of-interest QP if the immediately-preceding encoding amount is larger than the encoding amount that satisfies the target bit rate or decreases the non-region-of-interest QP if the immediately-preceding encoding amount is smaller than the encoding amount that satisfies the target bit rate.
In the second control method, the QP controller 811 uses past data about the quantization parameter and the encoding amount to generate a model in which the relationship between the quantization parameter and the encoding amount has been learned. By using the model, the QP controller 811 selects a quantization parameter that may generate an encoding amount that satisfies the target bit rate.
The following method is conceivable as a method of providing different quantization parameters to a region-of-interest and a non-region-of-interest.
FIG. 3 is a configuration diagram illustrating the internal configuration of a conventional image encoding system 90.
The image encoding system 90 in FIG. 3 fixes the quantization parameter of one of the region-of-interest and the non-region-of-interest and changes the other quantization parameter so as to perform rate control.
The image encoding system 90 has a block-of-interest determining unit 91 that outputs a block-of-interest including input pixels-of-interest, a rate controller 92 that decides a region-of-interest QP and a non-region-of-interest QP from the target bit rate and the fixed QP and outputs the region-of-interest QP and the non-region-of-interest QP, and a compressor 93 that compresses the input image in accordance with a compression method, such as JPEG, H.264, or H.265, based on the block-of-interest, the region-of-interest QP, and the non-region-of-interest QP and outputs a stream.
The rate controller 92 has a QP controller 921 that generates a non-region-of-interest QP from the target bit rate and outputs the non-region-of-interest QP.
Although the region-of-interest QP is fixed in this example, a variation in which the non-region-of-interest QP is fixed is also conceivable.
Although the quality of the region-of-interest or the non-region-of-interest is not guaranteed in the image encoding system 80 illustrated in FIG. 2 since the offset is the limiting condition for the quantization parameters, significant deviation of the qualities of the region-of-interest and the non-region-of-interest do not occur. In contrast, although the quality of the region-of-interest or the non-region-of-interest is guaranteed in the image encoding system 90 illustrated in FIG. 3, significant deviation of the qualities of the region-of-interest and the non-region-of-interest may possibly occur.
Although a detailed embodiment of the QP controller 921 is not clearly specified in particular, for example, a conceivable method involves performing control based on FB information output from the compressor 93, as illustrated in FIG. 3.
Specifically, the following two control methods are conceivable.
In the first control method, in the case where the region-of-interest QP is fixed and the non-region-of-interest QP is variable, as illustrated in FIG. 3, the QP controller 921 compares the encoding amount for the immediately-preceding frame with the target bit rate and increases the non-region-of-interest QP if the immediately-preceding encoding amount is larger than the encoding amount that satisfies the target bit rate or decreases the non-region-of-interest QP if the immediately-preceding encoding amount is smaller than the encoding amount that satisfies the target bit rate.
In the second control method, the QP controller 921 uses past data about the quantization parameter and the encoding amount to generate a model in which the relationship between the quantization parameter and the encoding amount has been learned. By using the model, the QP controller 921 selects a quantization parameter that may generate an encoding amount that satisfies the target bit rate.
It is desirable to reduce an excessive encoding amount or an insufficient encoding amount since it may cause situations such as loss of data in a buffer or a transmission path of a device or an inability to effectively utilize resources.